Video compression method and apparatus

ABSTRACT

A video compression system may have first and second dual-port memory devices, a third memory device, and first and second processors that may provide enhanced processing, including motion estimation. The first processor may be configured to store in the second memory device first and second video frames and to transfer sequential sets of data from the first video frame corresponding to fields of search. A second set of a plurality of adjacent macroblocks of the second video frame may be compared to macroblocks selected from the field of search. Dual-port memory devices may allow for the concurrent use of shared memory by the two processors as well as data transfer during processing.

RELATED APPLICATION

This is a division of application Ser. No. 10/404,952, filed Mar. 31,2003,U.S. Pat. No. 7,519,115, which is incorporated herein by referencein its entirety for all purposes.

BACKGROUND

The present disclosure relates to video signal processing, such as videocompression.

Digital video is the format commonly used with personal computers,digital-video cameras, and other electronic systems. Since a huge amountof memory or storage space is required to fully store all 30 or moreframes per second of video, the images are usually compressed. Oftensequential images in the video sequence differ only slightly. Thedifference from a previous (or following) image in the sequence can bedetected and encoded, rather than the entire picture using a compressiontechnique, such as MPEG encoding.

MPEG is a video signal compression standard, established by the MovingPicture Experts Group (“MPEG”) of the International StandardizationOrganization. MPEG is a multistage algorithm that integrates a number ofwell known data compression techniques into a single system. Theseinclude motion-compensated predictive coding, discrete cosine transform(“DCT”), adaptive quantization, and variable length coding (“VLC”). Themain objective of MPEG is to remove redundancy that normally exists inthe spatial domain (within a frame of video) as well as in the temporaldomain (frame-to-frame), while allowing inter-frame compression andinterleaved audio.

There are two basic forms of video signals: an interlaced scan signaland a non-interlaced scan signal. An interlaced scan signal is atechnique employed in television systems in which every television frameconsists of two fields referred to as an odd-field and an even-field.Each field scans the entire picture from side to side and top to bottom.However, the horizontal scan lines of one (e.g., odd) field arepositioned half way between the horizontal scan lines of the other(e.g., even) field. Interlaced scan signals are typically used inbroadcast television (“TV”) and high definition television (“HDTV”).Non-interlaced scan signals are typically used in computer systems andwhen compressed have data rates up to 1.8 Mb/sec for combined video andaudio. The Moving Picture Experts Group has established an MPEG-1protocol intended for use in compressing/decompressing non-interlacedvideo signals, and an MPEG-2 protocol intended for use incompressing/decompressing interlaced TV and HDTV signals.

Before a conventional video signal may be compressed in accordance witheither MPEG protocol it must first be digitized. The digitizationprocess produces digital video data which specifies the intensity andcolor of the video image at specific locations in the video image thatare referred to as pixels. Each pixel is associated with a coordinatepositioned among an array of coordinates arranged in vertical columnsand horizontal rows. Each pixel's coordinate is defined by anintersection of a vertical column with a horizontal row. In convertingeach frame of video into a frame of digital video data, scan lines ofthe two interlaced fields making up a frame of un-digitized video areinterdigitated in a single matrix of digital data. Interdigitization ofthe digital video data causes pixels of a scan line from an odd-field tohave odd row coordinates in the frame of digital video data. Similarly,interdigitization of the digital video data causes pixels of a scan linefrom an even-field to have even row coordinates in the frame of digitalvideo data.

MPEG-1 and MPEG-2 each divides a video input signal, generally asuccessive occurrence of frames, into sequences or groups of frames(“GOF”), also referred to as a group of pictures (“GOP”). The frames inrespective GOFs are encoded into a specific format. Respective frames ofencoded data are divided into slices representing, for example, sixteenimage lines. Each slice is divided into macroblocks each of whichrepresents, for example, a 16×16 matrix of pixels. Each macroblock isdivided into six blocks including four blocks relating to luminance dataand two blocks relating to chrominance data. The MPEG-2 protocol encodesluminance and chrominance data separately and then combines the encodedvideo data into a compressed video stream. The luminance blocks relateto respective 8×8 matrices of pixels. Each chrominance block includes an8×8 matrix of data relating to the entire 16×16 matrix of pixels,represented by the macroblock. After the video data is encoded it isthen compressed, buffered, modulated and finally transmitted to adecoder in accordance with the MPEG protocol. The MPEG protocoltypically includes a plurality of layers each with respective headerinformation. Nominally each header includes a start code, data relatedto the respective layer and provisions for adding header information.

There are generally three different encoding formats which may beapplied to video data. Intra-frame coding produces an “I” block,designating a block of data where the encoding relies solely oninformation within a video frame where the macroblock of data islocated. Inter-frame coding may produce either a “P” block or a “B”block. A “P” block designates a block of data where the encoding relieson a prediction based upon blocks of information found in a prior videoframe. A “B” block is a block of data where the encoding relies on aprediction based upon blocks of data from surrounding video frames,i.e., a prior I or P frame and/or a subsequent P frame of video data.

One means used to eliminate frame-to-frame redundancy is to estimate thedisplacement of moving objects in the video images, and encode motionvectors representing such motion from frame to frame. The accuracy ofsuch motion estimation affects the coding performance and the quality ofthe output video. Motion estimation performed on a pixel-by-pixel basishas the potential for providing the highest quality video output, butcomes at a high cost in terms of computational resources. Motionestimation can be performed on a block-by-block basis to providesatisfactory video quality with a significantly reduced requirement forcomputational performance.

These techniques are used for reducing the data required to store videosignals, or for transmitting video signals over communication linkshaving a smaller bandwidth than is required to transmit uncompressedvideo. Examples of such communication links includes local areanetworks, wide area networks, and circuit-switched telephone networks,such as integrated services digital network (ISDN) lines or standardtelephone lines.

Video signal processing and video signal compression are variouslydescribed in Video Demystified: A Handbook for the Digital Engineer,Second Ed., by K. Jack, High Text Interactive, Inc., San Diego, Calif.,U.S.A., 1996; Image and Video Compression Standards Algorithms andArchitectures, Second Edition, by V. Bhaskaran et al., Kluwer AcademicPublishers, Norwell Mass., U.S.A., 1997; Algorithms, Complexity Analysisand VLSI Architectures for MPEG-4 Motion Estimation, by P. Kuhn, KluwerAcademic Publishers, Dordrecht, The Netherlands, 1999; as well as inU.S. Pat. Nos. 6,421,466 B1; 6,363,117; 6,014,181; 5,731,850; and5,510,857; and U.S. Patent Application Publication Nos. 2002/0176502 A1;and 2002/0131502 A1, all of which are incorporated in this descriptionby reference.

BRIEF SUMMARY

A method of video compression may be practiced by storing in a firstmemory device data representative of at least a portion of a first videoframe. Sets of data representative of corresponding portions of theframe may then be transferred sequentially from the first memory deviceto a second memory device. Each set of data stored in the second memorydevice may then be processed according to a video compression process.During processing of each set of data stored in the second memorydevice, a sequentially next set of data from the first memory device istransferred to the second memory device.

A video compression system may include first and second memory devicesand one or more processors for performing a method of compressing videodata. The processor(s) may be configured to store in the first memorydevice data representative of at least a portion of a first video frame,and to transfer sequential sets of data representative of correspondingportions of the frame from the first memory device to the second memorydevice. The processor(s) may then process each current set of datastored in the second memory device according to the video compressionprocess while transferring a sequentially next set of data from thefirst memory device to the second memory device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a general block diagram of a video compression system.

FIGS. 2A and 2B form a combined block diagram of an embodiment of thesystem of FIG. 1.

FIG. 3 is a general schematic of an alternative embodiment of aconcurrent read/write memory that may be used in the system of FIG. 1.

FIG. 4 is a diagram illustrating a search algorithm associated with afield of search.

FIG. 5 is an enlarged illustration of a portion of the search fieldillustrated in FIG. 4.

FIG. 6 is a flow chart of a video-compression process.

FIG. 7 is a flow chart of another video-compression process.

FIG. 8 is a further flow chart of yet another video-compression process.

DETAILED DESCRIPTION

As has been mentioned, video compression may be performed with multipleunit processing. This multi-unit processing may take various forms. Inone form, it provides rapid serial processing of video compressionfunctions. As an example, FIG. 1 illustrates a block diagram of a videocompression system 10. System 10 includes a stage N processor 12 thatreceives video data from what is generally referred to as an inputdevice 14. Device 14 may be any source of data, such as a communicationmedium or link, such as a cable or bus, a memory device, buffer,register, processor or other data functional or storage device.Processor 12 may be any process that processes video information, suchas a processor that performs one or more functions relating to, forexample, motion estimation, motion compensation, discrete-cosinetransformation, quantization, and entropy encoding.

Processor 12 processes data received from input device 14 and writes orotherwise stores the processed data in a concurrent read/write memorydevice 16. Data may be written into memory device 16 at the same time asdata previously stored in the device is read out. That is, reading andwriting of data may occur concurrently. Any device, apparatus orcombination of devices that provide this function may, in the generalsense be used. Two examples of device 16 are illustrated in FIGS. 2A, 2Band 3, which are described below. Accordingly, data previously recordedin device 16 by processor 12 may be read into a stage N+1 processor 18.Processor 18 may process video data that has been processed previouslyby processor 12. For instance, processor 12 may perform motionestimation and compensation, and processor 18 may perform discretecosine transform (DCT) and quantization in a video compression system.Other examples are given in the system illustrated in FIGS. 2A and 2B.

Once the processed data received by the stage N+1 processor is furtherprocessed, the data is output to what is generally referred to as anoutput device 20. Similar to device 14, output device 20 may be anydestination for data, such as a communication medium or link, a memorydevice, buffer, register, processor or other device for processing,storing or transmitting data.

It will be appreciated then that processors 12 and 18 and intermediatememory provide a system that may allow for rapid transfer of databetween the two processors while making the processors nearlyindependent. In an exemplary video compression application, processor 12may be performing an Nth video compression process on an (N+1)th blockof video data and progressively writing processed data out to memorydevice 16 during the processing. While processor 12 is writing data intomemory device 16, processor 18 may be reading data associated with anNth block of data that was previously processed by processor 12 andstored in the memory device. Processors 12 and 14 may thereby be able tofunction on the respective blocks of data without having to depend on orinteract with the operation of the other processor. Each processingfunction may thereby be internally optimized.

A more detailed example of a video compression system 30 is illustratedin FIGS. 2A and 2B. These two figures together provide a general blockdiagram of a video compression system that may incorporate the featuresof system 10 just described. System 30 includes an input device 32,which may be a SDI interface that provides demultiplexing of multiplexedvideo/audio data, and stores, for example, 32 lines of video data in adual-port RAM. The processing of the demultiplexed audio data isprovided by conventional means and is not further discussed.

Included in system 30 is a motion estimator 34, a motion compensator 36,a DCT and quantization (DCT/Q) processor 38, an entropy encoder 40, andan output device 42. Output device 42 provides multiplexing and dataselection to produce an output compressed video signal 44. A systemfeedback processor 46 may receive processing information from the motioncompensator, DCT/Q processor and entropy encoder for controlling therate of processing at each stage and the amount of data being generatedat each stage. The feedback system may modify the operation of theprocessors in order to normalize the rate and quantity of coded datageneration by system 30 so that the output video signal may maintain atarget level of data output. Other than as described, these variousfunctional processors may function conventionally, and furtherexplanation is not provided.

Motion estimator 34 may receive a digital video signal 48 from inputdevice 32 in the form of successive 16×16 pixel macroblocks. System 30may process a slice of 16 lines of video at a time. Estimator 34includes a P frame motion estimator 50 and a B-frame motion estimator52. In applications where B frames are not determined, the B-framemotion estimator may not be used. In applications where only I framesare used, motion compensation would not be required. The I frames may bepassed through the motion estimator and compensator without processing.

Referring now to P-frame estimator 50, successive luma macroblock datais input to a coarse luma search processor 54. Coarse processor 54 iscoupled to a dual-port RAM 56 that may store an entire search area, alsoreferred to as a field of search, of data of a previously processedreference I frame. RAM 56 may receive field of search data from anexternal DDR SDRAM 58 that may store data for four frames. Processor 54may provide for transfer of data to RAM 56 and SDRAM 58, but when astate machine 60 or other processor provides this function, thefunctional requirements of processor 54 may be reduced. Accordingly, onemay refer to a general processor 64 that includes the functionality ofprocessors 54 and 60.

SDRAM 58 is referred to as an external device because a single chip 62may include all of the structure shown for system 30, except for theexternal DDR-SDRAM's. An example of such a chip is a field-programmablegate array (FPGA) sold under the proprietary name of Xilinx® Virtex-II®,available from Xilinx, Inc. of San Jose, Calif., U.S.A.

As is explained further below with reference to FIG. 4, the courseprocessor 54, as part of a hierarchical search, may select a best matchfor each macroblock or group of current macroblocks of a P frame, forwhich motion estimation is being performed, relative to a referencefield of search. There are various known algorithms that may be used forselecting a best match. One such method is the computation of the sum ofthe absolute differences (SAD) between a current macroblock and areference macroblock. The reference macroblock that has the lowest SADvalue may then be considered to be the best match. The results areoutput to a dual-port RAM 66. Data may by read into RAM 66 at the sametime that previously stored data is read out of it. In this case, dataassociated with a previous coarse search is read from RAM 66 by a fineluma search processor 68.

For each given current macroblock, processor 68 may perform a furthersearch in more detail in a reduced field of search centered on the bestmatch identified in the previous stage of the motion estimation. Thefield of search may be a portion of the field of search used in thecoarse search. This field of search data is also read out of RAM 56.Since RAM 56 is a dual-port RAM, processor 68 may access RAM 56 whileprocessor 54 is accessing RAM 56. This allows for simultaneous datatransfer from a single memory device and relatively independentfunctioning of the processors.

A replacement best match may be found within this reduced field ofsearch and the results passed on to another dual-port RAM 70. Previouslystored data is output from RAM 70 to a block difference processor 72forming part of motion compensator 36. Processor 72 may compute a motionvector based on the position differences between each current macroblockand the associated best-fit reference macroblock determined duringmotion estimation. This motion vector is based on luma values.Differences between the chroma values for each pair of current andreference macroblocks is also determined. The chroma values may beobtained from an external DDR SDRAM 74 having stored chroma valuescorresponding to the frames for which SDRAM 58 stores luma values. Thedifference values are written into a dual-port RAM 76.

B-frame motion estimator 52 includes elements that are mirror images ofelements contained in P-frame motion estimator 50. Accordingly,estimator 52 includes a coarse search processor 78, dual-port RAM 80storing the current field of search of a reference frame and an externalDDR SDRAM 82. A dual-port RAM 84 couples search processor 78 with a finesearch processor 86. The output of processor 86 is stored in a dual-portRAM 88. A block difference processor 90 of motion compensator 36 readsdata stored in RAM 88 and in an external DDR SDRAM 92. The blockdifference data is written into a dual-port RAM 94.

A final difference block 96 reads data from both RAM's 76 and 94. Thereason for this is that frames treated as a B-frames have motionestimation determined by B-frame motion estimator 52, and also byP-frame motion estimator 50, as though the frame was a P-frame. Finaldifference block 96 compares the results of the two motion estimationand compensation processes and determines which one provides the bettermatch between the current frame and the respective reference frame. Theone with a better match is used and the other is disregarded.

Dual-port RAM's also provide interfaces between the remaining stages ofvideo compression system 30. A RAM 98 is disposed between processors 96and 38, and a RAM 100 is disposed between processors 38 and 40.

Entropy encoder 40 includes an entropy encoding processor 102 that iscoupled to registers 104, 106 and 108 that provide, respectively, headerinformation, DC values and AC values for data to be transmitted. Thecompressed video data and associated components of a compressed videosignal are transmitted to data select processor 42 for production of thedata stream that becomes video signal 44 transmitted over acommunication link to a video receiver.

Concurrent read/write memory devices may be in the form of the dual-portRAM's illustrated in FIG. 2. Additionally, they may formed of acombination of components that provide for concurrent reading andwriting. Such a memory device is shown generally at 110 in FIG. 3.Memory device 110 couples a stage N processor 112 to a stage N+1processor 114. Processor 112 outputs data D_(IN) to an address A_(IN).Processor 114 inputs data D_(OUT) received from an address A_(OUT). Amultiplexer 116 receives the data D_(IN) and outputs it to one of outputlines D_(IN1), and D_(IN2), based on a control signal 118 received froma state machine (not shown) based on a control signal 120 output fromprocessor 112. The multiplexer writes successive sets of dataalternately to a RAM 1 and a RAM 2. The address lines from processors112 and 114 are input to a router 122. The router outputs a receivedinput address to either an address line A1 connected to RAM 1 or to anaddress line A2 connected to RAM 2 based on a received control signal124. Each of RAM 1 and RAM 2 either read received data or write storeddata based on respective control signals 126 and 128. Data is read outfrom RAM 1 and RAM 2 on respective data lines D_(OUT1), and D_(OUT2)connected to inputs on a multiplexer 130. This multiplexer then outputsthe data received on either of these data lines on data line D_(OUT)based on a control signal 132. The operation of stage N+1 processor 114is coordinated with the operation of stage N processor by a controlsignal 134.

Memory device 110, in the general sense, allows processor 112 to writedata to one RAM while processor 114 reads data from the other RAM, andboth RAM's may receive data from processor 112 and may output data toprocessor 114. However, because the address lines must be coordinated,as shown, both processors may not address both RAM's at the same time.This configuration provides for separate functioning of the twoprocessors and their operations do not require that one be completedbefore the other can begin.

Referring now to FIG. 4, coarse processor 54 may use a coarse field ofsearch as shown generally at 140. This discussion is directedspecifically to P-frame motion estimator 50, although it may beequivalently applied to B-frame motion estimator 52. The size of thefield of search may be based on the amount of time required to read thedata into dual-port memory 56 and the amount of time it takes to conductthe search. The complete field of search 140 may be stored in thedual-port memory 56 for direct access by processor 54.

In determining motion estimation for a P frame, the reference frame isan I frame. Field 140 is shown as an array of seven rows by twentycolumns. The columns may be considered as five groups of four columnseach. The columns in each group are designated as columns A, B, C and D.The array may thus be considered to be an array of sets of four adjacentmacroblocks. For instance, a group 142 of four macroblocks in column 5of the array includes reference macroblocks designated R_(A)(5,3),R_(B)(5,3), R_(C)(5,3) and R_(D)(5,3). In the center of the array arefour adjacent macroblocks identified as C_(A), C_(B), C_(C) and C_(D).Macroblocks C_(A), C_(B), C_(C) and C_(D) are not part of array 140, butrather form a set 144 of macroblocks of a current frame for which motionestimation is being determined. The macroblocks in current set 144 havepositions in the current frame corresponding to positions R_(A)(3,4),R_(B)(3,4), R_(C)(3,4) and R_(D)(3,4) of array 140. That is, a field ofsearch is selected, in this case, that is +/−3 rows of macroblocksvertically and +/−2 columns of four-macroblock sets horizontally.

The macroblocks in current set 144 may each be compared concurrently toeach of the macroblocks shaded as shown. This is a summary form ofdesignation. As is well known in the art, one macroblock is compared toanother macroblock by comparing corresponding pixel values in bothmacroblocks. The shaded macroblocks correspond to every other macroblockin every other row. Other search strategies, such as every othermacroblock in every row or different search field configurations, may beused depending on the requirements of a particular application. Further,the search field may take configurations other than a rectangular array.Table I below illustrates the steps in a coarse motion estimation searchfor four macroblocks C_(A), C_(B), C_(C) and C_(D).

TABLE I COARSE SEARCH MB STEP C_(A) CB C_(C) C_(D) 1 R_(A)(1, 1)R_(A)(1, 1) R_(A)(1, 1) R_(A)(1, 1) 2 R_(C)(1, 1) R_(C)(1, 1)R_(C)(1, 1) R_(C)(1, 1) 3 R_(A)(3, 1) R_(A)(3, 1) R_(A)(3, 1)R_(A)(3, 1) 4 R_(C)(3, 1) R_(C)(3, 1) R_(C)(3, 1) R_(C)(3, 1) . . . . .. . . . . . . . . . 37  R_(A)(5, 5) R_(A)(5, 5) R_(A)(5, 5) R_(A)(5, 5)38  R_(C)(5, 5) R_(C)(5, 5) R_(C)(5, 5) R_(C)(5, 5) 39  R_(A)(7, 5)R_(A)(7, 5) R_(A)(7, 5) R_(A)(7, 5) 40  R_(C)(7, 5) R_(C)(7, 5) R_(C)(7,5) R_(C)(7, 5)

It is seen that each selected reference macroblock R(J,K) is comparedconcurrently to each of the four macroblocks in current group 144. Thesesteps continue until each of the current macroblocks are compared toeach of the selected reference macroblocks. As has been mentioned, thecomparison may be of a minimum function, such as the minimum sum of theabsolute differences for each pair of macroblocks compared. As a resultof the coarse search, a best match is determined for each currentmacroblock. The best match may be different for the four currentmacroblocks. For instance, reference macroblock R_(C)(3,1) may be thebest match for macroblock C_(A), and reference macroblock R_(A)(5,5) maybe the best match for macroblock C_(D).

Once the best matches are selected by coarse processor 54, the resultsare stored in RAM 66. Processor 54 then proceeds to perform the samecoarse search for the next four adjacent current macroblocks. Finesearch processor 68 may be processing the previous set of four adjacentcurrent macroblocks while processor 54 is performing a search for set144. Processor 68 stores the results of its search in RAM 70 and thenreads in from RAM 66 the results of the coarse search on current set144. A different field of search is applied to the fine search. In thisexample, the field of search is a 3×3 macroblock square array, such asarray 150 shown in FIG. 5. Array 150 as a result is a 48×48 pixel array.Other sizes of the field of search may be used. Array 150 may becontained within array 140 and may be centered on the position of thebest fit macroblock R_(C) associated with a current macroblock C. Whenarray 150 is contained in array 140, the data is directly accessiblefrom RAM 56. Further, since RAM 56 is a dual-port RAM, processors 54 and68 may access the data concurrently, thereby making the data availablefrom a single memory device. As an example of an array 150 and referringagain to the example in FIG. 4, a fine field-of-search array 152associated with current macroblock C_(A) may be centered aroundreference macroblock R_(C)(3,1).

Rather than compare the current macroblock with alternate macroblocks, afiner or more dense, search is performed. Every macroblock embedded inthe reduced array may be searched or fewer macroblocks may be searched,depending on the allocated time for computing the best match. As anexample, Table II below illustrates the steps that may be used forperforming a fine search of reduced array 150. In this table, eachcurrent macroblock C_(X) is compared concurrently to a set of referencemacroblocks R_(X)(J,K). Each reference macroblock is designated by thelocation of the upper left pixel. In the example shown, macroblocksidentified by alternate pixel locations in pixel rows and columns aresearched.

TABLE II FINE SEARCH STEP MB 1 2 . . . 256 257 . . . 1024 C_(A) R_(A)(2,2) R_(A)(2, 10) R_(A)(32, 26) C_(A) R_(A)(2, 4) R_(A)(2, 12) R_(A)(32,28) C_(A) R_(A)(2, 6) R_(A)(2, 14) R_(A)(32, 30) C_(A) R_(A)(2, 8)R_(A)(2, 16) R_(A)(32, 32) C_(B) R_(B)(2, 2) C_(B) R_(B)(2, 4) . . .C_(B) R_(B)(2, 6) C_(B) R_(B)(2, 8) C_(C) C_(C) . . . C_(C) C_(C) C_(D)R_(D)(32, 26) C_(D) . . . R_(D)(32, 28) C_(D) R_(D)(32, 30) C_(D)R_(D)(32, 32)

The comparisons for the set of four current macroblocks are performedsequentially, since each one may be associated with a differentreference macroblock. A new reference macroblock may be selected afterthe fine search process that has a lower SAD than the referencemacroblock identified in the coarse search. Referring to FIG. 5, thereference macroblock identified in the coarse search corresponds inposition to macroblock R(16,16). A new reference macroblock, such asmacroblock R(6,26) may have the lowest SAD after the fine searchprocess. This information is output to dual-port RAM 70.

It is seen that at the general functional level, the dual-port RAM'sallow for concurrent use of a memory device by two sequentially adjacentprocessors, thereby permitting them to operate relatively independently.This gives the individual processors flexibility in functioning, havinglittle dependency on the ongoing function of adjacent processors.

A further aspect of motion estimator 50 is that field-of-search data isfed into dual-port RAM 56 from SDRAM 58 while processors 54 and 68 areprocessing data. Since a next set of current macroblocks may have afield of search that overlaps with that of a current set, it may only benecessary to read in, during processing of a given set of currentmacroblocks, that data required for the next set. This additional datais illustrated by partial array 140′ shown in FIG. 4. Thus, whenprocessing of a current set N is complete, the data for the field ofsearch for set N+1 has been entered, and processing on set N+1 may beginimmediately.

Referring now to FIG. 6, a method is shown generally at 160 in asimplified form for purposes of illustration. Method 160 may be directedto a method of sequential processing using an intermediate memorydevice. Beginning the method at 162, an index N may be initialized toN=0 at 164. The processing path then splits into two paths.

In the left path, the index N is incremented by 1 at 166. An N^(th) setof video data is processed according to a first video compressionprocess, such as those processes illustrated in system 30 shown in FIG.2. The processed N^(th) set of video data is written into a memorydevice. A determination is made at 170 whether an (N−1)^(th) set hasbeen read from the memory device. If it has not, further processing maybe delayed at 172 to allow an increment of additional time to lapse. Thedetermination at 170 is then repeated, and this cycle repeats until the(N−1)^(th) set has been read. At that time, a determination is made at174 as to whether there is more data to process. If not, processing isended at 176. Otherwise, processing is continued and the index is againincremented at 166 and the process repeated.

In the right path, the index N is incremented by 1 at 178. An (N−1)^(th)set of video data is read at 180 from the memory device and processedaccording to a second video compression process. A determination is thenmade at 182 whether an N^(th) set has been written into the memorydevice. If so, processing is continued and the index is againincremented at 178 and the process repeated. If not, a determination ismade at 184 as to whether there is more data to process. If not,processing is ended at 186. If there is more data, further processingmay be delayed at 188 to allow an increment of additional time to lapse.The determination at 182 is then repeated, and this cycle repeats untilthe N^(th) set of data has been written.

The respective steps of processing data and storing it in the memorydevice at 168, and reading the stored data and processing it at 180 mayoccur at the same time. Further, these processes may be independent ofeach other except with regard to the coordinating of the reading andwriting of sequential sets of data into the memory device. The processesmay also be sequential in that one set of data is first processed andthen passed on to the second process step via the memory device forfurther processing. This sequential processing further allows therespective process steps to be internally optimized.

A second method is shown generally at 200 in FIG. 7. Method 200 may bedirected to changing data in a memory device to allow relativelyuninterrupted processing of the changing data. Once the method begins at202, data representative of a first video frame is stored in a firstmemory device at 204. An index N is initialized to zero at 206 and thenthe method divides into two paths.

In the right path, the index N is incremented at 208. An N^(th) set ofdata not included in a previously stored (N−1)^(th) set of data may betransferred from the first memory device to a second memory device at210. A determination may then be made at 212 as to whether the(N−1)^(th) set of data has been processed. If it has not, furtherprocessing may be delayed at 214 to allow an increment of additionaltime to lapse. The determination at 212 is then repeated, and this cyclerepeats until the (N−1)^(th) set has been processed. Once it has beenprocessed, a determination may be made at 216 as to whether there ismore data. If so, processing is continued at 208 and the index isincremented at step 210 and the subsequent steps repeated. If there isno more data, the method ends at 218.

In the left path, the index N is incremented by 1 at 220. An (N−1)^(th)set of video data is read at 222 from the second memory device andprocessed according to a video compression process. A determination maythen be made at 224 whether an N^(th) set has been transferred into thesecond memory device. If so, processing is continued and the index isagain incremented at 220 and the process repeated. If not, adetermination is made at 226 as to whether there is more data toprocess. If not, processing is ended at 228. If there is more data,further processing may be delayed at 230 to allow an increment ofadditional time to lapse. The determination at 224 is then repeated, andthis cycle repeats until the N^(th) set of data has been written intothe second memory device.

The respective steps of transferring data into the second memory deviceat 210 and reading the stored data and processing it at 222 may occur atthe same time. Further, these processes may be independent of each otherexcept with regard to the coordinating of the reading and writing of thedata into the second memory device. The processes may be performed onsequential sets of data in that one set of data is first transferred tothe second memory device and then the stored data is processed. Thisprocessing of sequential sets of data may also allow these respectiveprocess steps to be internally optimized.

Referring now to FIG. 8, yet another method, shown generally at 240, isshown. Method 240 may be directed to performing motion estimation on aplurality of adjacent macroblocks of a current frame with concurrentprocessing of a plurality of macroblocks. The method may begin at 242followed by storing, at 244, in a first memory device, data for a fieldof search of a first video frame corresponding to a second set ofadjacent macroblocks of a second video frame. The first video frame maybe a reference frame, such as an I frame or a P frame. The second videoframe, referred to as a current frame, may be a P frame or a B frame,depending on the motion estimation process being used.

A first set of macroblocks may be selected from the field of search at246. At 248, a plurality of macroblocks of one of the first and secondsets may be compared concurrently with at least one macroblock of theother set. A determination may then made at 250 as to whether all of thesecond set has been compared to the first set. If so, a determinationmay be made at 252 as to whether there is more data. If not, the processmay be ended at 254. Otherwise, processing may return to step 244 for anew field of search. If it is determined in step 250 that all of thesecond set has not been compared, then at least one of a differentplurality of macroblocks and a different one macroblock may be selectedat 256. Processing is then returned to step 248 and the processcontinued.

By processing concurrently a plurality of macroblocks, motion estimationmay occur at a very rapid rate. Further, by providing motion estimationof a plurality of adjacent macroblocks of a current video frame, motionestimation may be further expedited, as compared to processing onecurrent-frame macroblock at a time.

Although several processors have been identified separately in thisdescription, these processors may be combined or even further separatedinto various other combinations. Separate processors may provide forconcurrent processing of data.

The preceding description is presented largely in terms of diagrams,algorithms, and symbolic representations of structure and processoroperation. These descriptions and representations may be implemented anddescribed as various interconnected distinct software modules,structures or features. This is not necessary, as software, firmware,and hardware may be configured many different ways, and may beaggregated into a single processor and program with unclear boundaries.Program modules, executed by one or more computers or other devices,include routines, programs, objects, components, data structures thatperform particular tasks or implement particular abstract data types.The functionality of program modules may be combined or distributed asdesired in various embodiments.

An algorithm is generally considered to be a self-consistent sequence ofsteps leading to a desired result. These steps require physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities may take the form of electrical or magnetic signalscapable of being stored, transferred, combined, compared, and otherwisemanipulated. As a convention, these signals may be referred to as bits,values, elements, symbols, characters, images, terms, numbers, or thelike. These and similar terms may be associated with appropriatephysical quantities and are convenient labels applied to thesequantities.

Processes realizable in the form of computer programs may be stored inany computer-readable medium. Computer-readable media may be anyavailable media that may be accessed by a computer. By way of example,computer-readable media may comprise volatile and non-volatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer-readableinstructions, data structures, program modules, or other data.Computer-readable media may further include RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical storage medium, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium that may be used to store information and that may be accessed bya computer.

The present disclosure also relates to apparatus for performingoperations. This apparatus may be specially constructed for the requiredpurposes or it may comprise a general-purpose computer selectivelyactivated or reconfigured by a computer program stored in the computeror other apparatus. In particular, various general-purpose machines maybe used with programs in accordance with the teachings described, or itmay prove more convenient to construct more specialized apparatus toperform the required method steps.

The programs described need not reside in a single memory, or even asingle machine. Various portions, modules or features of them may residein separate memories, or even separate machines. The separate machinesmay be connected directly, or through a network, such as a local accessnetwork (LAN), or a global network, such as what is presently known asthe Internet™.

The above description is intended to be illustrative, and notrestrictive. Many other embodiments will be apparent to those of skillin the art upon reviewing the above description. The scope of a claimedinvention should, therefore, be determined with reference to the claims,along with the full scope of equivalents to which such claims areentitled. Accordingly, while embodiments of video compression systemsand video-compression methods have been particularly shown anddescribed, many variations may be made therein. This disclosure mayinclude one or more independent or interdependent inventions directed tovarious combinations of features, functions, elements and/or properties,one or more of which may be defined in the following claims. Othercombinations and sub-combinations of features, functions, elementsand/or properties may be claimed later in this or a related application.Such variations, whether they are directed to different combinations ordirected to the same combinations, whether different, broader, narroweror equal in scope, are also regarded as included within the subjectmatter of the present disclosure. An appreciation of the availability orsignificance of claims not presently claimed may not be realized at thetime of filing of this disclosure. Accordingly, the foregoingembodiments are illustrative, and no single feature or element, orcombination thereof, is essential to all possible combinations that maybe claimed in this or a later application. Each claim defines aninvention disclosed in the foregoing disclosure, but any one claim doesnot necessarily encompass all features or combinations that may beclaimed. Where the claims recite “a” or “a first” element or theequivalent thereof, such claims include one or more such elements,neither requiring nor excluding two or more such elements. Further,ordinal indicators, such as first, second or third, for identifiedelements are used to distinguish between the elements, and do notindicate a required or limited number of such elements, and do notindicate a particular position or order of such elements unlessotherwise specifically stated.

1. A video compression system comprising: a first memory device; a firstprocessor configured to process sequential sets of video data accordingto a first video compression process, and to write processed data to thememory device; and a second processor configured to read video dataprocessed by the first video compression process from the memory devicewhile the first processor is writing processed data to the memorydevice, and process sequential sets of the read data by a second videocompression process; the first processor being further configured towrite a third set of processed data over a first set of processed datawhile the second processor reads a second set of processed data.
 2. Thesystem of claim 1, where the first processor is configured to processsets of data with each set including a plurality of adjacent blocks ofdata, and to perform at least one process step on each block of data. 3.The system of claim 2, where the first processor is further configuredto write into the memory device a set of processed data prior toprocessing a subsequent set of data.
 4. A video compression systemcomprising: a first memory device; a second memory device; and at leasta first processor configured to store in the first memory device datarepresentative of at least a portion of a first video frame, to transfersequential sets of data representative of corresponding portions of theframe from the first memory device to the second memory device, toprocess each current set of data stored in the second memory deviceaccording to a first video compression process while transferring asequentially next set of data from the first memory device to the secondmemory device.
 5. The system of claim 4, where the processor isconfigured to process sequential sets of overlapping data in which thesequentially next set of data overlaps with the current set of data, theprocessor being further configured to read from the first memory thatportion of the next sequential set of data not included in the currentset of data during processing of the current set of data.
 6. The systemof claim 5, where the video compression process is motion estimation,the processor is configured to store in the first memory device at leastportions of each of a plurality of video frames, and the sets of dataare fields of search.
 7. The system of claim 5, further comprising asecond processor configured to read data stored in the second memorydevice and to process a portion of each current set of data stored inthe second memory device according to a second video compressionprocess.
 8. The system of claim 7, where the first and second processorsare configured to read concurrently data from the second memory device.9. A method of compressing video data comprising: processing accordingto a first video compression process sequential sets of video data witheach set including a plurality of adjacent blocks of data, includingprocessing each block in each set of data; writing a first set of videodata processed by the first video compression process to a memorydevice; writing a second set of video data processed by the first videocompression process to the memory device; while writing the second setof data processed by the first video compression process, reading thefirst set of video data from the memory device; processing the read databy a second video compression process; writing a third set of dataprocessed by the first video compression process over the first set ofdata processed by the first video compression process; and reading asecond set of data processed by the first video compression processduring writing of the third set of data.
 10. The method of claim 9,where processing the video data includes processing sequential sets ofdata with each set including a plurality of adjacent blocks of data,including processing each block in each set of data.
 11. The method ofclaim 10, where writing into the memory device the first set ofprocessed data is prior to processing the subsequent second set of videodata by the first video compression process.
 12. A method of compressingvideo data comprising: storing in a first memory device datarepresentative of at least a portion of a first video frame;transferring sequential sets of data representative of correspondingportions of the frame from the first memory device to a second memorydevice; processing each set of data stored in the second memory deviceaccording to a first video compression process; and during processing ofeach set of data stored in the second memory device, transferring asequentially next set of data from the first memory device to the secondmemory device.
 13. The method of claim 12, where processing data storedin the second memory device includes processing sequential sets ofoverlapping data in which a sequentially next set of data overlaps witha current set of data, and reading from the first memory that portion ofthe next sequential set of data not included in the current set of dataduring processing of the current set of data.
 14. The method of claim13, where processing each set of data includes estimating motion bysearching in a field of search, and storing data in the first memorydevice includes storing at least portions of each of a plurality ofvideo frames, and transferring sets of data includes transferring setsof data corresponding to fields of search.
 15. The method of claim 13,further comprising processing a portion of each current set of datastored in the second memory device according to a second videocompression process.
 16. The method of claim 15, where processing eachset of data stored in the second memory device includes reading a firstportion of data from the second memory device, and processing a portionof each current set of data stored in the second memory device includesreading a second portion of data from the second memory deviceconcurrently with reading the first portion of data from the secondmemory device.
 17. A computer-readable medium readable by one or moreprocessors and having embodied therein a program of computer-readableinstructions that, when executed by the one or more processors, providefor: processing according to a first video compression processsequential sets of video data with each set including a plurality ofadjacent blocks of data, including processing each block in each set ofdata; writing a first set of video data processed by the first videocompression process to a memory device; writing a second set of videodata processed by the first video compression process to the memorydevice; while writing the second set of data processed by the firstvideo compression process, reading the first set of video data processedby the first video compression process from the memory device;processing the read data by a second video compression process; writinga third set of data processed by the first video compression processover the first set of data processed by the first video compressionprocess; and reading the second set of data processed by the first videocompression process during writing of the third set of data.
 18. Acomputer-readable medium readable by one or more processors and havingembodied therein a program of computer-readable instructions that, whenexecuted by the one or more processors, provide for: storing in a firstmemory device data representative of at least a portion of a first videoframe; transferring sequential sets of data representative ofcorresponding portions of the frame from the first memory device to asecond memory device; processing each set of data stored in the secondmemory device according to a first video compression process; and duringprocessing of each set of data stored in the second memory device,transferring a sequentially next set of data from the first memorydevice to the second memory device.
 19. The computer-readable medium ofclaim 18, where processing data stored in the second memory deviceincludes processing sequential sets of overlapping data in which asequentially next set of data overlaps with a current set of data, andreading from the first memory that portion of the next sequential set ofdata not included in the current set of data during processing of thecurrent set of data.
 20. The computer-readable medium of claim 19, whereprocessing each set of data includes estimating motion by searching in afield of search, and storing data in the first memory device includesstoring at least portions of each of a plurality of video frames, andtransferring sets of data includes transferring sets of datacorresponding to fields of search.
 21. The computer-readable medium ofclaim 19, where the instructions further provide for processing aportion of each current set of data stored in the second memory deviceaccording to a second video compression process.
 22. Thecomputer-readable medium of claim 21, where processing each set of datastored in the second memory device includes reading a first portion ofdata from the second memory device, and processing a portion of eachcurrent set of data stored in the second memory device includes readinga second portion of data from the second memory device concurrently withreading the first portion of data from the second memory device.